Support java-test-fixtures plugin #223

pkubowicz · 2020-07-27T17:03:12Z

Applying 'java-test-fixtures' plugin caused 'No mutations found' from Pitest.

The plugin changes test runtime classpath: tested code is included as a JAR instead of as a directory
(see gradle/gradle#11696). It seems that Pitest ignores correct directories being passed using --mutableCodePaths and insists on having those directories also passed in --classPath.

Also change functional tests: runTasksSuccessfully() fails the test without giving any context, so it should be avoided.

The project I work on uses java-test-fixtures plugin, so this issue blocks me from using Pitest.

pkubowicz · 2020-07-27T17:17:33Z

Builds failing on Travis seems to be victims of the unstable build. My PR passes on Travis for Java 11 and 14. Tests pass when I run them locally with Java 8. AppVeyor build with Java 8 passes.

szpak

Thanks for your PR.

The build fails only with Java 8, as PIT 1.1.5 is not treated as Java 11 compatible and:

PitestPluginPitVersionFunctionalSpec.setup and run pitest task with PIT 1.1.5 FAILED

isn't executed in the other builds. As PIT 1.4.0 was the one to required Java 8, feel free to remove any older PIT versions from the verification matrix.

I put some comment/questions/suggestions inline in code.

src/funcTest/groovy/info/solidsoft/gradle/pitest/functional/Junit5FunctionalSpec.groovy

szpak · 2020-08-02T19:35:25Z

src/funcTest/groovy/info/solidsoft/gradle/pitest/functional/Junit5FunctionalSpec.groovy

@@ -21,8 +23,10 @@ class Junit5FunctionalSpec extends AbstractPitestFunctionalSpec {
        given:
            copyResources("testProjects/junit5kotlin", "")
        when:
-            ExecutionResult result = runTasksSuccessfully('pitest', '-b', 'build.gradle.kts')
+            ExecutionResult result = runTasks('pitest')


Copy-paste? You probably would like to build using the kotlin build file :).

src/funcTest/groovy/info/solidsoft/gradle/pitest/functional/Junit5FunctionalSpec.groovy

szpak · 2020-08-02T19:37:11Z

src/funcTest/groovy/info/solidsoft/gradle/pitest/functional/TestFixturesFunctionalSpec.groovy

+@CompileDynamic
+class TestFixturesFunctionalSpec extends AbstractPitestFunctionalSpec {
+
+    void "should work with kotlin and test fixtures"() {


kotlin and test fixtures? That project seem to have classes written in Java and build file in Groovy :).

Yes, it's a copy-paste mistake, I will fix it.

szpak · 2020-08-02T19:38:12Z

src/main/groovy/info/solidsoft/gradle/pitest/PitestPlugin.groovy

        task.additionalClasspath.setFrom({
            List<FileCollection> testRuntimeClasspath = (extension.testSourceSets.get() as Set<SourceSet>)*.runtimeClasspath
            FileCollection combinedTaskClasspath = project.objects.fileCollection().from(testRuntimeClasspath)
            FileCollection filteredCombinedTaskClasspath = combinedTaskClasspath.filter { File file ->
-                !extension.fileExtensionsToFilter.getOrElse([]).find { extension -> file.name.endsWith(".$extension") }
-            }
+                !extension.fileExtensionsToFilter.getOrElse([]).find { extension -> file.name.endsWith(".$extension") || file == "core.jar" }


Why core.jar is special?

I forgot to remove it after debugging a local project.

szpak · 2020-08-02T19:40:47Z

src/test/groovy/info/solidsoft/gradle/pitest/PitestTaskConfigurationSpec.groovy

-        return [new File(project.buildDir, "classes//java//${sourceSetName}"),
-                new File(project.buildDir, "resources//${sourceSetName}")
+        return [new File(project.buildDir, "classes/java/${sourceSetName}"),
+                new File(project.buildDir, "resources/${sourceSetName}")


I believe that I had some problems (somewhere) with just a single / on Windows and // was fine on both *Unix and Windows. However, the tests passed on AppVeyor, so it doesn't seem to be a problem here...

szpak · 2020-08-02T19:48:39Z

src/test/groovy/info/solidsoft/gradle/pitest/PitestTaskConfigurationSpec.groovy

+            task.taskArgumentMap()['classPath'] ==
+                (
+                    assembleSourceSetsClasspathByNameAsStringSet("intTest") +
+                    [new File(project.buildDir, "classes/java/main").absolutePath]


I wonder that it started to be needed here. classpath didn't contain have classes/java/main before, assuming java plugin was applied? Or it was a limitation of this "unit" test (and in real execution it was included)?

intTest source set is created from scratch and does not have any relation to main source set, so intTest classpath does not include any classes from main. I suppose previously such a project won't work with real Pitest, because Pitest would fail to find classes to mutate.

I know what confused me.

extension.mainSourceSets.set([javaSourceSets.getByName(SourceSet.MAIN_SOURCE_SET_NAME)])

and:

private Set<File> calculateBaseMutableCodePaths() { Set<SourceSet> sourceSets = extension.mainSourceSets.get() return sourceSets*.output.classesDirs.files.flatten() as Set<File> }

which you call should cover it. However, in the past it was used only for mutableCodePaths, not for (additional)classpath, what you fixed making it java-test-fixtures plugin compatible :).

Btw, could you, making other changes, refactor assembleSourceSetsClasspathByNameAsStringSet to generate class only or class+resources classpath to keep new File(project.buildDir, "classes/java/XXX") in one place?

I'm not sure what you wanted, but I changed the path handling.

Please check and close other discussions, I think I addressed all your comments.

Sort of :). I will tune it up after merge.

szpak · 2020-08-02T20:26:43Z

Btw, I spotted that the weekly cron build on Travis hasn't started "recently" and the problem with PIT 1.1.5 started to occur also in devel.

I will fix it right away in devel and let you know to rebase your branch.

Due to integration test failures with recent Java 8 versions: #223 (review)

szpak · 2020-08-02T20:37:40Z

Building. Feel free to rebase once it's green.

Applying 'java-test-fixtures' plugin caused 'No mutations found' from Pitest. The plugin changes test runtime classpath: tested code is included as a JAR instead of as a directory (see gradle/gradle#11696). It seems that Pitest ignores correct directories being passed using --mutableCodePaths and insists on having those directories also passed in --classPath. Also change functional tests: runTasksSuccessfully() fails the test without giving any context, so it should be avoided.

pkubowicz · 2020-08-05T12:45:50Z

Is there something this PR is waiting for to be merged?

szpak · 2020-08-05T17:30:10Z

Nope, I accepted it and it's waiting for a longer moment of time to adjust the last commit after merge :).

pkubowicz · 2020-08-18T13:29:22Z

Can you release this bugfix and refactor tests later? There is no workaround for this problem.

szpak · 2020-08-18T17:10:03Z

Thanks for a reminder. 1.5.2 is just being released on Travis:
https://travis-ci.org/github/szpak/gradle-pitest-plugin/jobs/719031199

pkubowicz · 2020-08-18T18:24:59Z

Thank you!

pkubowicz force-pushed the test-fixtures branch from 8a5c9c5 to b3ec5df Compare July 27, 2020 17:21

szpak requested changes Aug 2, 2020

View reviewed changes

szpak added a commit that referenced this pull request Aug 2, 2020

Bump minimal support PIT version to 1.4.0

1d78bcd

Due to integration test failures with recent Java 8 versions: #223 (review)

pkubowicz force-pushed the test-fixtures branch from b3ec5df to 608e731 Compare August 3, 2020 06:27

Refactor paths in PitestTaskConfigurationSpec

327bae2

szpak approved these changes Aug 5, 2020

View reviewed changes

szpak added this to the 1.5.2 milestone Aug 18, 2020

szpak merged commit 0747c03 into szpak:master Aug 18, 2020

pkubowicz deleted the test-fixtures branch August 18, 2020 18:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support java-test-fixtures plugin #223

Support java-test-fixtures plugin #223

pkubowicz commented Jul 27, 2020 •

edited

Loading

pkubowicz commented Jul 27, 2020 •

edited

Loading

szpak left a comment

szpak Aug 2, 2020

szpak Aug 2, 2020

pkubowicz Aug 2, 2020

szpak Aug 2, 2020

pkubowicz Aug 2, 2020

szpak Aug 2, 2020

szpak Aug 2, 2020

pkubowicz Aug 3, 2020

szpak Aug 4, 2020

szpak Aug 4, 2020

pkubowicz Aug 5, 2020

szpak Aug 5, 2020

szpak commented Aug 2, 2020

szpak commented Aug 2, 2020

pkubowicz commented Aug 5, 2020

szpak commented Aug 5, 2020

pkubowicz commented Aug 18, 2020

szpak commented Aug 18, 2020 •

edited

Loading

pkubowicz commented Aug 18, 2020

Support java-test-fixtures plugin #223

Support java-test-fixtures plugin #223

Conversation

pkubowicz commented Jul 27, 2020 • edited Loading

pkubowicz commented Jul 27, 2020 • edited Loading

szpak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

szpak commented Aug 2, 2020

szpak commented Aug 2, 2020

pkubowicz commented Aug 5, 2020

szpak commented Aug 5, 2020

pkubowicz commented Aug 18, 2020

szpak commented Aug 18, 2020 • edited Loading

pkubowicz commented Aug 18, 2020

pkubowicz commented Jul 27, 2020 •

edited

Loading

pkubowicz commented Jul 27, 2020 •

edited

Loading

szpak commented Aug 18, 2020 •

edited

Loading